When Formal Systems Kill: Computer Ethics and Formal Methods

Computer ethics is usually motivated by and deals with problematic decision-making cases arising from the ubiquity and unprecedented power of computer and information technologies. In this paper, we address a distinct topic that is germane to computer ethics but has thus far been neglected in the computer ethics literature. Formal methods is the science of mathematically proving computer systems correct. Formal methods are motivated by the need for safety-critical systems (e.g., autopilot systems) that must be bug-free. We begin by describing the automatic formal system property of computers. This property has been studied by cognitive scientists, but it has yet to be addressed by computer ethicists, despite it having significant ethical ramifications, particularly with respect to formal methods. After introducing the history and purview of formal methods, we argue that normative questions about formal methods are the purview of computer ethics. Nevertheless, we take the position in this paper that these questions are open questions and do not attempt to solve these problems here.